Blur Kernel Research Articles

Turbulent image deblurring using a deblurred blur kernel

Abstract In the context of addressing a noisy turbulence-degraded image, it is common to use a denoising low-pass filter before implementing a deblurring algorithm. However, this filter not only suppresses noise but also induces a certain degree of blur into the degraded image. This blur effect causes a blurred estimate of the true blur kernel and ultimately leads to a distorted estimate of the latent clear image. To tackle this issue, this paper presents an innovative single-image deblurring method. It integrates a dedicated blur kernel deblurring step to mitigate the effects of the denoising filter. The L0 norm and L2 norm serve as the respective constraints for latent clear image and blur kernel. Experimental results on both synthetic and real-world turbulence-degraded images demonstrate the effectiveness and efficiency of the proposed method.

Image denoising and model-independent parameterization for IVIM MRI.

Objective: To improve intravoxel incoherent motion imaging (IVIM) magnetic resonance Imaging (MRI) quality using a new image denoising technique and model-independent parameterization of the signal versus b-value curve.Approach: IVIM images were acquired for 13 head-and-neck patients prior to radiotherapy. Post-radiotherapy scans were also acquired for five patients. Images were denoised prior to parameter fitting using neural blind deconvolution, a method of solving the mathematical problem of blind deconvolution using neural networks. The signal decay curve was then quantified in terms of several area under the curve (AUC) parameters. Improvements in image quality were assessed using blind image quality metrics, total variation (TV), and the strength of correlations between IVIM parameters in parotid glands with radiotherapy dose levels. The accuracy of blur kernel predictions was assessed by the ability to recover artificial "pseudokernels" which had been applied to denoised images. AUC parameters were compared with apparent diffusion coefficient (ADC), biexponential, and triexponential model parameters in terms of their correlations with dose, contrast-to-noise (CNR) ratios inside and outside parotid glands, and their relative importance as determined via principal component analysis.
Main Results: Image denoising improved blind image quality metrics, smoothed the signal versus b-value curve, and strengthened correlations between IVIM parameters and dose. Image TV was reduced and parameter CNRs generally increased following denoising. AUC parameter changes had higher correlations with dose and higher relative importance than traditional IVIM parameters.Significance: IVIM parameters have high variability in the literature and perfusion-related parameters are difficult to interpret. Describing the signal versus b-value curve with model-independent parameters like the AUC and preprocessing images with denoising techniques could potentially benefit IVIM image parameterization in terms of reproducibility and functional utility.

Non-local degradation modeling for spatially adaptive single image super-resolution

Existing methods for single image super-resolution (SISR) model the blur kernel as spatially invariant across the entire image, and are susceptible to the adverse effects of textureless patches. To achieve improved results, adaptive estimation of the degradation kernel is necessary. We explore the synergy of joint global and local degradation modeling for spatially adaptive blind SISR. Our model, named spatially adaptive network for blind super-resolution (SASR), employs a simple encoder to estimate global degradation representations and a decoder to extract local degradation. These two representations are fused with a cross-attention mechanism and applied using spatially adaptive filtering to enhance the local image detail. Specifically, SASR contains two novel features: 1) a non-local degradation modeling with contrastive learning to learn global and local degradation representations, and 2) a non-local spatially adaptive filtering module (SAFM) that incorporates the global degradation and spatial-detail factors to preserve and enhance local details. We demonstrate that SASR can efficiently estimate degradation representations and handle multiple types of degradation. The local representations avoid the detrimental effect of estimating the entire super-resolved image with only one kernel through locally adaptive adjustments. Extensive experiments are performed to quantitatively and qualitatively demonstrate that SASR not only performs favorably for degradation estimation but also leads to state-of-the-art blind SISR performance when compared to alternative approaches.

Unsupervised Pan-Sharpening via Mutually Guided Detail Restoration

Pan-sharpening is a task that aims to super-resolve the low-resolution multispectral (LRMS) image with the guidance of a corresponding high-resolution panchromatic (PAN) image. The key challenge in pan-sharpening is to accurately modeling the relationship between the MS and PAN images. While supervised deep learning methods are commonly employed to address this task, the unavailability of ground-truth severely limits their effectiveness. In this paper, we propose a mutually guided detail restoration method for unsupervised pan-sharpening. Specifically, we treat pan-sharpening as a blind image deblurring task, in which the blur kernel can be estimated by a CNN. Constrained by the blur kernel, the pan-sharpened image retains spectral information consistent with the LRMS image. Once the pan-sharpened image is obtained, the PAN image is blurred using a pre-defined blur operator. The pan-sharpened image, in turn, is used to guide the detail restoration of the blurred PAN image. By leveraging the mutual guidance between MS and PAN images, the pan-sharpening network can implicitly learn the spatial relationship between the two modalities. Extensive experiments show that the proposed method significantly outperforms existing unsupervised pan-sharpening methods.

Robust Blind Text Image Deblurring via Maximum Consensus Framework

The blind text image deblurring problem presents a formidable challenge, requiring the recovery of a clean and sharp text image from a blurry version with an unknown blur kernel. Sparsity-based strategies have demonstrated their efficacy by emphasizing the sparse priors of the latent image and kernel. However, these existing strategies have largely neglected the influence of additional noise, imposing limitations on their performance. To overcome this limitation, we propose a novel framework designed to effectively mitigate the impact of extensive noise prevalent in blurred images. Our approach centers around a robust Maximum Consensus Framework, wherein we optimize the quantity of interest from the noisy blurry image based on the maximum consensus criterion. Furthermore, we propose the integration of the Alternating Direction Method of Multipliers (ADMM) and the Half-Quadratic Splitting (HQS) method to address the computationally intractable L0 norm problem. This innovative strategy enables improvements in the deblurring performance of blurry text images with the additional synthetic noise. Experimental evaluations conducted on various noisy blurry text images demonstrate the superiority of the proposed approach over existing methods.

Neural blind deconvolution for deblurring and supersampling PSMA PET.

Objective. To simultaneously deblur and supersample prostate specific membrane antigen (PSMA) positron emission tomography (PET) images using neural blind deconvolution.Approach. Blind deconvolution is a method of estimating the hypothetical 'deblurred' image along with the blur kernel (related to the point spread function) simultaneously. Traditionalmaximum a posterioriblind deconvolution methods require stringent assumptions and suffer from convergence to a trivial solution. A method of modelling the deblurred image and kernel with independent neural networks, called 'neural blind deconvolution' had demonstrated success for deblurring 2D natural images in 2020. In this work, we adapt neural blind deconvolution to deblur PSMA PET images while simultaneous supersampling to double the original resolution. We compare this methodology with several interpolation methods in terms of resultant blind image quality metrics and test the model's ability to predict accurate kernels by re-running the model after applying artificial 'pseudokernels' to deblurred images. The methodology was tested on a retrospective set of 30 prostate patients as well as phantom images containing spherical lesions of various volumes.Main results. Neural blind deconvolution led to improvements in image quality over other interpolation methods in terms of blind image quality metrics, recovery coefficients, and visual assessment. Predicted kernels were similar between patients, and the model accurately predicted several artificially-applied pseudokernels. Localization of activity in phantom spheres was improved after deblurring, allowing small lesions to be more accurately defined.Significance. The intrinsically low spatial resolution of PSMA PET leads to partial volume effects (PVEs) which negatively impact uptake quantification in small regions. The proposed method can be used to mitigate this issue, and can be straightforwardly adapted for other imaging modalities.

Blind multi-Poissonian image deconvolution with sparse log-step gradient prior.

Blind image deconvolution plays a very important role in the fields such as astronomical observation and fluorescence microscopy imaging, in which the noise follows Poisson distribution. However, due to the ill-posedness, it is a very challenging task to reach a satisfactory result from a single blurred image especially when the power of the Poisson noise is at a high level. Therefore, in this paper, we try to achieve high-quality restoration results with multi-blurred images which are contaminated by Poisson noise. Firstly, we design a novel sparse log-step gradient prior which adopts a mixture of logarithm and step functions to regularize the image gradients and combine it with the Poisson distribution to formulate the blind multi-image deconvolution problem. Secondly, we incorporate the methods of variable splitting and Lagrange multiplier to convert the original problem into sub-problems, then we alternately solve them to achieve the estimation of all the blur kernels of corresponding blurred images. Besides, we also design a non-blind multi-image deconvolution algorithm which is based on the log-step gradient prior to reach the final restored image. Experimental results on both synthetic and real-world blurred images show that the proposed prior is very capable of suppressing negative artifacts caused by ill-posedness. The algorithm can achieve restored image of very high quality which is competitive with some state-of-the-art methods.

Object recognition via echoes: quantifying the crossmodal transfer of three-dimensional shape information between echolocation, vision, and haptics.

Active echolocation allows blind individuals to explore their surroundings via self-generated sounds, similarly to dolphins and other echolocating animals. Echolocators emit sounds, such as finger snaps or mouth clicks, and parse the returning echoes for information about their surroundings, including the location, size, and material composition of objects. Because a crucial function of perceiving objects is to enable effective interaction with them, it is important to understand the degree to which three-dimensional shape information extracted from object echoes is useful in the context of other modalities such as haptics or vision. Here, we investigated the resolution of crossmodal transfer of object-level information between acoustic echoes and other senses. First, in a delayed match-to-sample task, blind expert echolocators and sighted control participants inspected common (everyday) and novel target objects using echolocation, then distinguished the target object from a distractor using only haptic information. For blind participants, discrimination accuracy was overall above chance and similar for both common and novel objects, whereas as a group, sighted participants performed above chance for the common, but not novel objects, suggesting that some coarse object information (a) is available to both expert blind and novice sighted echolocators, (b) transfers from auditory to haptic modalities, and (c) may be facilitated by prior object familiarity and/or material differences, particularly for novice echolocators. Next, to estimate an equivalent resolution in visual terms, we briefly presented blurred images of the novel stimuli to sighted participants (N = 22), who then performed the same haptic discrimination task. We found that visuo-haptic discrimination performance approximately matched echo-haptic discrimination for a Gaussian blur kernel σ of ~2.5°. In this way, by matching visual and echo-based contributions to object discrimination, we can estimate the quality of echoacoustic information that transfers to other sensory modalities, predict theoretical bounds on perception, and inform the design of assistive techniques and technology available for blind individuals.

Weber's Law-based Regularization for Blind Image Deblurring

Blind image deblurring aims to recover an output latent image and a blur kernel from a given blurred image. Kernel estimation is a significant step in blind image deblurring and requires a regularization technique to minimize the cost function and the edges of objects to generate a sharp image in a better way. This study proposes a new image regularization technique called Weber's Law Regularization (WLR) based on the Weber law phenomenon. The Weber ratio was used to preserve the edges of small salient objects and to minimize the cost function to obtain a sharp image while minimizing the ringing effect. To validate the WLR, experiments were conducted on benchmark synthetic and real word images and compared with existing state-of-the-art methods. The experimental results showed that WLR can effectively and efficiently deblur images even in the absence of prior knowledge.

Inertial sensor aided motion blur kernel estimation for cooled IR detector

Motion blur happens due to the variation in scene depth and the relative motion between the target and the imaging sensor. There are two types of infra-red (IR) imager: i) Cooled IR Imager and ii) Uncooled IR Imager. Images taken by these IR imagers exhibit various types of motion blur pattern. These different types of motion blur patterns are directly caused by the difference in working principle of detector. We suggest a novel method to generate synthetic blurry images from sharp images by analyzing the mechanisms of image formation on the cooled IR detector. We also suggest a novel blur kernel rendering technique for cooled IR imager using the inertial sensor information. The efficacy of the proposed blur kernel rendering technique is validated by deblurring the real blurry images captured from cooled IR camera. For evaluation, we construct a dataset of synthetic blurry images based on acquired sharp thermal images. This dataset is the first blurry thermal image dataset, for cooled IR imager, with ground-truth images. Extensive qualitative and quantitative experiments carried out on our dataset shows that our proposed method outperforms the contemporary analytical methods and upscale the deblurring performance of gyroscope aided deep learning methods.

Restoration of Radiographic Neutron Image Using Single-Channel Blind Deconvolution

This paper addresses the challenge of accurately estimating the 2D Point Spread Function (PSF) or blur kernel in neutron radiography, where traditional methods, such as the Edge Spread Function (ESF), prove time-consuming and reliant on manual edge selection. The proposed alternative introduces a robust sparse image prior known as the enhanced Patch-Wise Intensity (EPI) image prior in a singlechannel blind deconvolution algorithm, avoiding the need for intricate devices like pinhole or slit phantoms. Leveraging regularization and optimization techniques, the algorithm efficiently estimates PSF in a single image through a multilayer iterative alternating approach. The study aims to enhance PSF accuracy, leading to a more accurate solution to the neutron image restoration problem. Comparative results with real neutron images indicate the proposed method outperforms ESF, demonstrating improved overall image quality both visually and quantitatively in terms of blind/no reference evaluation (BRISQUE).

A comparative study of game theory techniques for blind deconvolution

The aim of this study is to lay emphasis on the potential of the use of Game theory to deal with Blind image Deconvolution. We consider a static game of two players. Player one controls the image intensity while the player two controls the blur kernel. In this game each player aims at minimizing his own functional. The outcome of the game is a pair of strategies: a deblurred image and an estimation of the blur kernel, that minimizes two functionals. We determine the optimal image deblurring using two particular game theoretic approaches, recently introduced: the Nash method [Meskine D., Moussaid N., Berhich S. Blind image deblurring by game theory. Proceedings of the 2nd International Conference on Networking, Information Systems & Security (NISS '19). 31 (2019)] and the Kalai–Smorodinsky solution method [Nasr N., Moussaid N., Gouasnouane O. The Kalai Smorodinsky solution for blind deconvolution. Computational and Applied Mathematics. 41, 222 (2022)]. We evaluate the performance of two techniques through numerical experiments and using some objective quality metrics.

Voice Vista

Blind image restoration involves leveraging prior information within an image to restore the sharpness of edges. De-blurring, on the other hand, aims to eliminate blurring artifacts caused by factors like defocus aberration or motion blur, which manifests as apparent streaking in still images of rapidly moving objects. Gaussian blur results from applying a Gaussian function to blur an image. Recent advancements in single-image methods owe their success in part to the utilization of various sparse priors for either latent images or motion blur kernels. KSR introduces an efficient kernel matrix approximation to accelerate blurring processes and achieve notable de-blur performance on digital datasets. License plate recognition serves as a critical tool for identifying over-speed vehicles or those involved in hit-and-run incidents. However, surveillance camera snapshots of speeding vehicles often suffer from motion blur, rendering them unrecognizable to the human eye and presenting a challenge to existing blind deblurring techniques. To address this, we propose a novel sparse representation-based scheme for identifying motion blur kernels. By analyzing sparse representation coefficients of the recovered image, we determine the kernel angle, and estimate the length of the motion kernel using Radon transform in Fourier domain. Our approach effectively handles large motion blur, even when license plates are unidentifiable by humans. Experimental evaluations on real-world images demonstrate the superiority of our method over several state-of-the-art blind image de-blurring algorithms.

SBHSR: single-band super-resolution method for hyperspectral images based on blind degradation and fusion of auxiliary band

Despite natural image super-resolution (SR) methods have achieved great success, super-resolution methods for hyperspectral image (HSI) with rich spectral features are still a very challenging task. Furthermore, due to the diversity of HSI captured by different cameras, their degradation conditions vary. Currently available HSI SR methods are mainly based on fixed degradation models, and their performance is severely affected when the actual degradation does not match the assumed degradation model. To address this issue, this paper proposes a single-band hyperspectral image super-resolution (SBHSR) method for HSI. This method assumes that the degradation Gaussian blur kernel parameters of the HSI are unknown, and adapts to various degradation conditions by conducting blind super-resolution on the image. It also combines another band with better spatial structure as an auxiliary band for feature fusion. To address the spectral differences between the super-resolution band and the auxiliary band, we use the Spatial Adaptation Module (SAM) to map the feature distribution, ensuring consistent spectral brightness between the auxiliary band and the super-resolution band. Due to the use of blind degradation super-resolution method, our method is more robust compared to previous HSI SR methods. Additionally, as it is specifically designed for single-band hyperspectral image super-resolution, it has the advantage of being faster and more efficient. Experimental validation on two hyperspectral datasets demonstrates the superiority of our method, as it improves the spatial resolution of the image while preserving the spectral features, and performs better than existing blind super-resolution methods.

Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution

Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.

An unrolled half-quadratic approach for sparse signal recovery in spectroscopy

This paper addresses the problem of sparse signal reconstruction frequently arising in analytical chemistry. Spectroscopic restoration is an important task that was tackled using two distinct approaches. First, model-based iterative methods were deployed. These methods rely on a strong theoretical background. However they face practical obstacles in terms of hyperparameter tuning. Conversely, approaches based on deep neural networks are fast and easy to implement. Nonetheless, they are often described as black-box tools, and their stability/robustness are ongoing research topics. In this paper, we propose a deep neural network based on unrolling a Majorization-Minimization (MM) Half-Quadratic (HQ) algorithm. This allows us to build interpretable layers mirroring iterations, making it possible to learn automatically data-driven hyperparameters such as regularization and stepsizes. Furthermore, we propose a dictionary of custom activation functions derived from potentials used in the original variational model. This interpretation of activations can be useful for analyzing the stability of neural networks. The efficiency of our method in comparison to iterative and learning-based methods is showcased through various experiments conducted on realistic mass spectrometry (MS) databases with various blur kernels and noise levels. All experiments show an improved reconstruction quality of chemical signals, while maintaining a low execution time at the test phase. All the results are reproducible, through the code shared on a repository available at https://github.com/GHARBIMouna/Unrolled-Half-Quadratic.

KGSR: A kernel guided network for real-world blind super-resolution

In recent years, deep learning-based methods have emerged as dominant players in the field of super-resolution (SR), owing to their exceptional reconstruction performance. The primary driver of their effectiveness lies in their utilization of extensive sets of paired low-resolution and high-resolution images for training deep learning models. This training enables the models to effectively replicate the intricate mapping relationship between low-resolution and high-resolution images. Nevertheless, at present, acquiring a sufficient quantity of such image pairs that satisfy the requirements remains a formidable obstacle. Therefore, in order to break the restriction of limited training sets, self-supervised learning has been introduced to train a model for each low-quality image, without requiring pairwise ground-truths. However, they generally presuppose the generation of low-resolution (LR) images from their high-resolution (HR) counterparts using a pre-defined kernel, such as Bicubic downscaling. Such an assumption is seldom valid for real-world LR images, where degradation processes in practical applications are diverse, intricate, and often undisclosed. Therefore, when the presumed downscaling kernel does not match the actual one, the outcomes of state-of-the-art approaches degrade substantially. In this paper, we introduce KGSR, a kernel-guided network for addressing real-world blind SR, effectively avoiding requiring large training image pairs and transforming the blind image super-resolution problem into a supervised learning and non-blind scenario. Specifically, KGSR trains two networks, namely Upscaling and Downscaling, utilizing only patches extracted from the input test image. On one hand, owing to the cross-scale recurrence property of the SR kernel within a single image, the Downscaling network acquires knowledge of the image-specific degradation process through a generative adversarial network. Consequently, the Downscaling network is capable of generating a downsampled version of the LR test image even when the acquisition process is unknown or less than ideal. Additionally, we employ a dedicated discriminator to compel the Downscaling network to prioritize the characterization of kernel orientations. Conversely, a precise blur kernel has the potential to yield superior performance. Guided by the accurate image-specific SR kernel acquired from the Downscaling network and the downsampled LR input, the Upscaling network is capable of producing a high-quality HR image from the LR input. Within the Upscaling network, we additionally introduce an effective module for harnessing the acquired image-specific SR kernel. KGSR operates as a fully unsupervised approach, yet it can concurrently produce both the image-specific SR kernel and high-quality HR images. Comprehensive experiments conducted on standard benchmarks validate the efficacy of the proposed approach compared to state-of-the-art methodologies. Moreover, the suggested method can deliver visually appealing SR outcomes while exhibiting shorter processing times when applied to real-world LR images.

Super-Resolution Degradation Model: Converting High-Resolution Datasets to Optical Zoom Datasets

Despite remarkable progress in single-image super-resolution based on neural networks, the results are not ideal when applied to real-world images, because the real-world degradation process is unknown and complex. The emergence of some optical zoom datasets shows that neural networks still achieve good results on real-world images as long as the low-resolution images used for training have similar features and distributions with the real-world images. However, obtaining such optical zoom datasets is complicated and the datasets are only applicable to specific cameras and shooting conditions. By studying the optical zoom datasets, we propose a super-resolution image degradation model consisting of blurring, frequency domain processing, adding noise and downsampling. Specifically, blurring uses a blur kernel with a wave-like shape inferred from the point spread function, which produces the artifacts like real-world images. Frequency domain processing simulates the frequency domain aliasing of real-world images, such as jagged edges and background stripes. Experiments demonstrate that the new degradation model achieves visual effects comparable to optical zoom datasets. Existing high-resolution datasets can be converted to “optical zoom datasets” by the degradation model, where the synthetic low-resolution images have real-world image features, thereby extending super-resolution methods to real-world images.

Kernel adaptive memory network for blind video super-resolution

Although recent video super-resolution (VSR) works show remarkable restoration performance for low-resolution (LR) video downscaled by a fixed known blur kernel, blind VSR suffers from severe performance degradation when the blur kernel is unknown. To alleviate this problem, blur kernel estimation methods have been proposed for VSR. However, existing VSR models must be trained separately with each LR dataset downscaled using all possible blur kernels. This is a time-consuming and memory-consuming task. To address these issues, we propose a kernel adaptive memory network for a blind VSR (KeMoVSR). The KeMoVSR mainly consists of a dual regression blur kernel estimator and a kernel adaptive VSR. The proposed blur kernel estimator predicts the parametric and non-parametric blur kernels by exploiting the blur kernel variation. Due to the blur kernel variation, the proposed kernel estimator can consider the temporal consistency of the blur kernel variation in adjacent frames, which leads to accurate blur kernel estimation in LR video frames. The proposed kernel adaptive VSR modulates the VSR weights according to the shape of the blur kernel by weight modulation layers. By integrating the proposed methods based on the memory network, we propose the KeMoVSR, which performs VSR by adaptively modulating the VSR weights using the blur kernel parameters as keys and values in memory networks. Experiments show that the KeMoVSR achieves superior performance compared to other blind VSR approaches. The KeMoVSR provides effective memory utilization that is appropriate for real-world scenarios. The code is available at https://github.com/dbseorms16/KeMoVSR.

A Fractional-Order Fidelity-Based Total Generalized Variation Model for Image Deblurring

Image deblurring is a fundamental image processing task, and research for efficient image deblurring methods is still a great challenge. Most of the currently existing methods are focused on TV-based models and regularization term construction; little efforts are paid to model proposal and correlated algorithms for the fidelity term in fractional-order derivative space. In this paper, we propose a novel fractional-order variational model for image deblurring, which can efficiently address three different blur kernels. The objective functional contains a fractional-order gradient fidelity term and a total generalized variation (TGV) regularization term, and it highlights the ability to preserve details and eliminate the staircase effect. To solve the problem efficiently, we provide two numerical algorithms based on the Chambolle-Pock primal-dual method (PD) and the alternating direction method of multipliers (ADMM). A series of experiments show that the proposed method achieves a good balance between detail preservation and deblurring compared with several existing advanced models.